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ABSTRACT 

The first issue of the Educational Policy Reform Research 
Institute’s (EPRRI ) publication, "Policy Updates," introduces the Institute 
and briefly reports on three key elements of accountability. It notes that 
EPRRI investigates the impact of educational accountability on students with 
disabilities and on special education. The Institute is conducting research 
in four core study states: California, Maryland, New York, and Texas. 
Examination of the first key element of accountability, federal policies 
supporting accountability, reviews new requirements under the Individuals 
with Disabilities Education Act (IDEA), the Government Performance Results 
Act, and Title I of the Leave No Child Behind Act. The following sections 
considers challenges in designing indicator systems for special education and 
offers 10 suggestions for choosing indicators. The final section reports 
findings of an ongoing analysis of 1999-2000 state assessment reports for 
students with disabilities. It finds that states have not yet fully complied 
with the IDEA directive concerning reporting of student assessment results 
and that reporting of alternate assessments is particularly rare. Recommended 
practices in reporting assessment results of students with disabilities are 
highlighted. (DB) 
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Introducing EPRRI 

The Educational Policy Reform Research Institute 



EPRRI, with funding from the U.S. 
Department of Education's Office of 
Special Education Programs, investi- 
gates the impact of educational 
accountability reforms on students 
with disabilities and on special edu- 
cation. EPRRI addresses the 
research needs of policy-makers and 
other key stakeholders by identifying 
critical gaps in current knowledge, 
seeking promising strategies, and 
publishing Topical Reviews, Policy 
Updates, and Issue Briefs. EPPRI is 
conducting research in four core study 
states: California; Maryland; New York; 
and Texas. 

The Institute is a joint venture of 
the Institute for the Study of 
Exceptional Children and Youth at 
the University of Maryland, the 
National Center on Educational 
Outcomes at the University of 
Minnesota, and the Urban Special 
^'^‘jcation Leadership Collaborative. 





EPRRI Reports on 

Three Key Elements of Accountabihty 



The demand for accountability for 
student performance is among the 
most visible and controversial of U.S. 
educational reforms. This EPRRI 
Policy Update highlights three 
important, interrelated elements of 
accountability for students with disabilities. 




Current federal require- 
ments and policies that 
support special education 
accountability. 



The challenges of 
creating special educa- 
tion indicators that can 
provide essential 
accountability 
information. 

How states are actually 
reporting assessment 
results for students with 
disabilities. 
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Federal Policies Supporting Accountability 



Public Reportings A 
Cornerstone of 
Accountability 

The most basic and visible form of 
accountability is the public report- 
ing of student test scores and 
other key education indicators 
such as graduation rates, drop out 
rates, and attendance. Coupled 
with these reports of student data 
are consequences - both sanctions 
and remedies - for schools and 
individual students. In addition, 
public reporting and account- 
ability can affect whom a school 
system hires; how it relates to 
students and parents; and how it 
manages resources, makes 
decisions, and ensures compliance 
with Federal regulations 
and requirements. 



EPRRrs Topical 

Review Creating 

Special Education 
Performance 
Goals and 
Indicators 
includes on 
overview of the Federal role in 
special education accountability 
systems. Here are selections from 
the overview (a downloadable version 
of the complete document is available at 
WWW. eprri. org/ products). 

New IDEA Accountability 
Requirements 



The 1 997 Individuals with 
Disabilities Education Act (IDEA) 
amendments contain a number of 
provisions to align federal special 
education policy with general educa- 
tion accountability reforms. 

Specifically, state education agen- 
cies must include students with disabil- 
ities in local and statewide assess- 
ments with accommodations where 
appropriate, and report the perform- 
ance of these students with the same 
frequency and in the same detail used 
to report non-disabled students' per- 
formance levels. 

States are also required to develop 
alternate means of assessment for stu- 
dents who are unable to participate in 
standard assessments, and the per- 
formance of these students must be 
reported as well. 

With these provisions, the law 
implicitly defines state and local 
assessments as contributing to a stu- 
dentis educational opportunities, for 
which access must be guaranteed. 
Tying progress to local and statewide 
assessments, with appropriate accom- 
modations, can ensure that students 
with disabilities will access the same 
standards as all other students. 
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New IDEA Requirements 
for Performance Goals 
and Indicators 

The 1 997 IDEA amendments 
require that states establish perform- 
ance goals and indicators for students 
with disabilities and report to the pub- 
lic on students' progress. The first 
Biennial Performance Reports for Part 
B of IDEA, due on December 31 , 
1999, were to identify the goals and 
performance indicators states would 
use to measure students' achievement 
on goals that, at a minimum, related 
to assessments, drop out rates, and 
graduation rates. 

A review of these reports indicates 
significant variation across states. 
What one state may have distin- 
guished as an indicator, another state 
identified as a goal. About one-third 
of the states reported that they 
applied the same set of goals to 
students in special education as in 
general education, while the remain- 
ing states had separate sets of goals 
for each group. 

Among all states, the most common 
goal addressed improving academic 
achievement, followed by improving 
transition or post-secondary place- 
ments, teacher preparation and tech- 
nical assistance, graduation rates, 
dropout rates, and communication or 
coordination with families and 
community (Ahearn, 2001 ). 

New Biennial Performance Report 
guidelines have been issued by Office 
of Special Education Programs (OSEP) 
and the second round of these 
reports are to be submitted by 
May 31, 2002. 
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GPRA and Title I 

The Government Performance Results 
Act (GPRA) holds federal agencies 
accountable for producing results, 
and links plans and outcomes with 
budgets. Within the Department of 
Education, OSEP has developed 
GPRA goals and indicators. 

Title I of the Leave No Child Behind 
Act, signed into law on January 8, 
2002 has sweeping new accountabili- 
ty requirements. Among the accounta- 
bility requirements are that each state 
develop and implement a statewide 
accountability system that ensures that 
all local school districts, and public 
elementary and secondary schools 
make Adequate Yearly Progress (AYP) 
on annual state assessments in read- 
ing and math (science will be added 
by 2005). As defined by the law. 
Adequate Yearly Progress applies the 
same high standards of academic 
performance to all public school 
students and measures the progress of 
public schools and local districts on 
state academic assessments and at 
district discretion, any academic or 

5 



other academic indicators. AYP 
includes annual measurable objectives 
for improvement of student perform- 
ance in each of the following groups 
of students: all public school students; 
economically disadvantaged students; 
students from major racial and ethnic 
groups; students with disabilities; and 
students with limited English proficiency. 

AYP must also include a timeline for 
ensuring that each group of students 
meets or exceeds a state determined 
"proficient" level of performance on 
the state assessment. (At a state's 
discretion, other academic measures 
such as promotion, high school com- 
pletion, and completion of college 
preparatory dasses nrroy also be included.) 

For a school to make adequate 
yearly progress, at least 95% of each 
of the subgroups of students must take 
the assessments. The low establishes 
a system of rewards and sanctions for 
states and local districts to hold public 
schools accountable for student 
achievement and making adequate 
yearly progress (ESEA, Part A, 

Subpart 1 , Sec.l 111). 
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States^ Annual Reports to 
Congress on IDEA 
Implementation 

States' Annual Reports to Congress 
on IDEA Implementation 
Since the implementation of P.L. 94- 
142, states have been required to 
report data annually on a variety of 
program indicators, which currently 
include the number of students receiv- 
ing special education services, their 
placements, the number of personnel 
serving them, and the exit status of 
those leaving school. The annual 
reports, which have been published 
since 1 978, are some of the main 
sources for information on the nation's 
progress implementing the IDEA. 




Challenges in Designing Indicator Systems 



The CIMP 

The Office of Special Education 
Program's Continuous Improvement 
Monitoring Process (CIMP) has been 
designed to assess "the impact and 
effectiveness of State and local efforts 
to provide early intervention services 
to infants and toddlers with disabilities 
and their families, and a free appro- 
priate public education to children 
and youth with disabilities" (OSEP, 
2000, p. 3). It employs a multi-step 
process engaging stakeholder commit- 
tees at local and state levels to assess 
critical program needs through the 
identification of key program indica- 
tors. Its focus is on using data to iden- 
tify improvement areas, and to design 
and evaluate improvements. 



EPRRI, in 
collabora- 
tion with the 
National 
Association 
of State 
Directors of 
Special 

Education (NASDSE), hosted a 
Policy Symposium in May 200 1 to 
discuss critical issues in creating a 
national special education indicator 
system that could be used for state 
accountability as well as school and 
system improvement efforts (proceed- 
ings of the symposium are available 
at www.eprri.org/policysymposia. 
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Participants at the Symposium 
included the special education direc- 
tors from each of the four core EPRRI 
study states and a number of local 
districts. Other important stakeholders 
such as disability advocates and par- 
ents, and representatives from nation- 
al education organizations and the 
U.S. Department of Education attend- 
ed. EPRRI is analyzing various per- 
formance data of the core states and 
local districts as well as reporting for- 
mats (available on our website 
www.eprri.org. 

State-level administrators and poli- 
cymakers identified the following indi- 
cators that they believe are for special 
education accountability and federal 
oversight: 

• Percent earning high school 
diploma 

• Percent dropping out of school 

• Participation in post-secondary 
education or employment 



® Classification and declassification 
rates disaggregated by 
ethnicity/ race 

• Measurement of Least Restrictive 
Environments 

• Number of complaints 
Local-level administrators and advo- 
cates expressed the need for a broad- 
er set of indicators that could be used 
to target school improvement efforts 
and keep parents informed about the 
status of the districts special education 
program. Local-level participants 
concluded that: 

• School level data must be the most 
comprehensive, because this is the 
level most sensitive to program 
improvement efforts. 

• Indicators need to reflect the diversi- 
ty of students with disabilities and 
their educational environments. 

• Student performance indicators 
should drive the accountability sys- 
tem, including those that measure 
social/behavioral status of students. 

• Important educational process indi- 
cators need to be collected, such as 
evaluations of parent participation, 
school improvement plans, staff 
evaluations, collaboration among 
professionals and alignment of admin- 
istrative and supervisory structures. 



® People in the trenches must under- 
stand the relevance of the data the 
are asked to collect. 

Choosing Indicators 

Indicators may differ depending on 
the stakes or consequences of deci- 
sions (e.g., resource allocation, 
capacity building, corrective action) 
and the level (student, school, district, 
state, or federal) at which they are 
made. If consequences, such as fur- 
ther investigation or sanctions, are 
attached to the data, the stakes 
increase and the properties of the 
indicators become even more critical. 

We offer here 10 practical consid- 
erations for selecting indicators that 
support valid decision-making: 

1 . All information is not of equal 
value. Indicators should be select- 
ed on the basis of their relevance 
to the specific decisions 

being made. 

2. The lower the level of inference 
required to interpret data associat- 
ed with an indicator, the more 
likely it will support valid 
decisions. 



3. Decisions are likely to be more 
valid when they are based on 
multiple sources and types of 
information that converge. 

4. Indicators that lack stability (over 
time, across groups, across set- 
tings, etc.) are of limited use for 
valid decision-making. 

5. Information loses value over time. 
To the extent possible, indicators 
should be selected that provide 
information in "real time" to 
decision-makers. 

6. Valid and reliable data, although 
they may cost more to collect, are 
needed for valid decision-making. 

7. Removing redundancies in data col- 
lection creates a more efficient system. 

8. Data collection should be coordi- 
nated both horizontally, across 
various programs and offices, and ver- 
tically, from school ta district ta state. 

9. Standardizing the operational 
definitions of specific indicators is 
important for comparability and 
coordination. 

1 0. Finally, all those who will use the 
indicators for decision making 
should value them. A statistic 
must have meaning and use, or 
people will have little incentive to 
collect and record accurately. 
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what Are Indicator 
Systems? 

Indicator systems are a key 
element of accountability, although 
they are not the whole accounto* 
bility system. Indicators provide 
information, which can range 
from student'level outcomes such 
as academic achievement, to 
school-level input such as teacher 
quality and class size. In an 
effective accountability system, the 
information that indicators offer is 
used to monitor the "health" of the 
education system, reported to 
appropriate agencies and the 
public, and acted upon. Among 
the indicators that states use most 
frequently are assessment scores, 
drop out rates, student 
attendance, and expenditures. 



To learn more 
about EPRRI and 
what we are 
doing. Visit our 
Web site at: 
www.eprri.org 




How States Report Assessment Results 



Reporting 
results for 
students 
with dis~ 
abilities 
helps 

ensure that 
these stu- 
dents are included in accountability 
systems. However, the form and 
content of reports vary widely 
among states and across school dis- 
tricts. The National Center on 
Educational Outcomes (NCEO's) 
ongoing analysis of 1 999-2000 
state assessment reports for students 
with disabilities indicates: 




• States have not yet fully complied 
with the IDEA directive to report on 
students with disabilities with the 
same frequency and detail as for 
other students; 

• Assessment results for many 
students are still not reported, 
including data for some students 
using non-approved accommoda- 
tions, using alternate assessments, 
or taking tests designed for lower 
grade levels; 

• Alternate assessments are new, and 
states face the technical challenge 
of aggregating these results with 
regular scores in a way that is 
statistically sound; 

• Only one state, Kentucky, has so 
far included information on 
alternate assessments; 

• As states track performance results 
over time, they need to document 
and account for changes in 
students' special education status. 
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Reporting assessment performance 
data is a critical step in education 
reform. The next important step is to 
use the information to improve out- 
comes and achievement levels for 
all students. 



Recommended 
Practices for State 
Reporting of 
Assessment Results of 
Students with 
Disabilities 

• Provide data from all test takers, 
whether they participate with or 
without accommodations or use 
an alternate assessment; 

• Give the rates of (as well as the 
reasons for) the exclusion of stu- 
dents with disabilities; 

• Keep records of accommoda- 
tions used; 

• Account for students whose 
scores are not aggregated with 
others, or who are not in the 
assessment system in any way; 

• Inform parents about the report- 
ing policy for their child's data; 

• Create reports that are clear, 
comprehensive, concise, and 
readily available; 

• Provide comparative information 
about schools, districts, states, 
regions, or standards, including 
changes over time; 

• Maintain confidentiality and 
offer cautions against misinter- 
pretation. 

Source: Nelson, Ysseldyke, & Thurlow, 1 998; 

Ysseldyke & Nelson, 1 998 
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EPRRI^ Core Study States 

We conduct research in collaboration 
with four states and eight local school 
districts: 

CALIFORNIA 

• Long Beach Unified School 
District 

• New Haven Unified School 
District 

MARYLAND 

• Carroll County Public Schools 

• Montgomery County 
Public Schools 

NEW YORK 

• North Colonie Central 
School District 

• Rochester City School District 
TEXAS 

• Cypress-Fairbanks Independent 
School Distrcit 

• Garland Independent 
School District 
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F@r M©re on These 
Policy Updofe TopieSoo^ 

Read EPRRI's Topical Reviews 
Creating Special Education 
Performance Goals and Indicators 
and Reporting on the State 
Assessment Performance of Students 
with Disabilities, available on 
www.eprri.org/products. You can 
also read about EPRRI's Policy 
Symposia atwww.eprri.org/ 
policysymposia. 

Who IS involved 

EPRRI is a project of the Institute for 
the Study of Exceptional Children and 
Youth at the University of Maryland, 
in collaboration with the National 
Center on Educational Outcomes and 
the Urban Special Education 
Leadership Collaborative. 



EPRRI is funded by the U.S. 
Department of Education's Office of 
Special Education Programs. 

The Insfifufe for the Study of 
Exceptional Children 
and Youth 

University of Maryland 
1308 Benjamin Building 
College Park, Maryland 20742-1 161 
301.405.6509 • fax: 301.314.9158 

National Center on 
Educational Outcomes 

University of Minnesota 
College of Education & 

Human Development 
350 Elliott Hall, 75 East River Road 
Minneapolis, MN 55455 
612.626.1530 • fax: 612.624.0879 

Urban Special Education 
Leadership Collaborative 

Education Development Center, Inc. 
55 Chapel Street 
Newton, MA 02458 
617.969.7100 • fax: 617.969.3440 
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U.S. Office of Special 
Education Programs 

The U.S. Department of Education's 

Office of Special Education Programs (OSEP) is 

committed to positive results for children with 

d‘"'hilities. The Institute is an IDEAs that Work project. 
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